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ABSTRACT 

This  thesis  proposes  a  system  failure  detection  method  —  Failure 
Projection  Method  (FPM)  which  provides  a  geometric  picture  of  the 
problem  of  failure  detection  in  the  presence  of  model  uncertainties 
and  noise. 

The  concept  of  FPM  is  thoroughly  studied  in  this  thesis. In 
particular,  two  groups  of  formulations  have  been  developed.  One  gives 
distinct  geometrical  interpretation  while  the  other  is  based  on  assuming 
that  one  has  available  a  priori  information  on  the  system  state.  Within 
two  groups  three  formulations  which  are  based  on  slightly  different 
criteria  and  have  decreasing  complexity  of  calculation  are  developed. 

The  simplest  require  only  a  singular  value  decomposition.  Also  two 
numerical  examples  are  given  which  show  the  relationship  among  these 
formulations  and  thus  provide  a  deeper  understanding  of  their  nature. 

An  algebric  approach  is  developed  for  the  generation  of  a  complete 
set  of  minimal  length  parity  checks. 

The  FPM  is  extended  to  including  measurement  and  process  noise. 
Again  a  forkulation  is  developed  which  only  involves  a  singular  value 
decomposition . 


k 


Again  a  formulation  is  developed  which  only  involves  a  singular  value 

decomposition. 

\ 

The  PPM  is  demonstrated  on  a  model  of  three  machine  power  system 
to  indicate  how  it  can  be  used  as  a  design  tool  in  assessing  system 
redundancy  and  in  determining  parity  checks. 
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CHAPTER  1 

INTRODUCTION 

1.1  Importance  of  Failure  Detection 

System  failure  detection  and  identification  are  important  issues 
for  which  a  great  amount  of  work  has  been  done  in  the  past  (see  the 
survey  in  [1] )  and  many  of  the  important  issues  in  failure  detection 
have  been  examined.  There  remain,  however,  some  important  problems 
to  be  examined  in  order  to  develop  useful  general  procedures.  One  of 
them  is  the  development  of  a  design  methodology  for  failure  detection 
algorithm  which  is  robust  to  model  uncertainties.  Up  to  the  present 
there  has  been  only  one  theoretical  effort  that  focussed  on  this 
problem.  This  is  the  work  of  Chow  [2]  and  Chow  and  Willsky  [3] .  Our 
research  uses  this  previous  work  as  a  starting  point  from  which  we 
shall  develop  significant  extensions  and  insights  into  this  impor¬ 
tant  problem. 

1.2  Key  Point  of  Failure  Detection 

As  discussed  by  Chow  [2],  the  failure  detection  process  can 
be  considered  as  consisting  of  two  stages:  residual  generation  and 
decision  making  based  on  the  residuals.  The  key  property  of  the 
residuals  is  that  they  should  behave  in  very  different  ways  for  each 
possible  failure  mode  so  that  an  accurate  decision  can  be  made.  That 
is,  we  desire  each  failure  to  have  a  distinct  signature  in  the  resi- 
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duals.  The  actual  residuals  will,  of  course,  deviate  from  the  signa¬ 
tures  because  of  the  presence  of  noise.  Also  the  actual  failure 
signatures  will  deviate  from  those  that  we  might  calculate  a  priori 
because  of  uncertainties  in  the  model.  The  design  of  robust  failure 
detection  systems  then  can  be  thought  of  in  the  following  way: 

(1.1)  Find  a  residual  generation  procedure  so  that  the  residuals 
will  be  small  if  there  is  no  failure  and  will  have  a  distinct  pattern 
if  there  is  a  failure.  This  should  be  true  in  the  presence  of  noise 
and  model  uncertainties. 

It  is  this  problem  that  we  will  address.  We  will  not  focus 
any  attention  on  the  decision-making  part  of  failure  detection,  where 
as  in  Chow's  work  [2],  we  think  of  the  robustness  issue  completely 
in  terms  of  providing  residuals  from  which  decision  making  is  made 
easier. 

as  mentioned  before,  one  important  issue  for  designing  a  residual 
generator  is  robustness,  i.e.  the  residual  generator  must  be  insensi¬ 
tive  to  system  modelling  errors  or  parameter  variations.  In  Chow's 
thesis  [2]  a  relationship  between  sensor  outputs  is  selected  so  that 
the  resulting  residual  will  be  as  small  as  possible  under  worst  case 
conditions  on  parameter  uncertainties  when  no  failure  has  ocurred. 

This  selection  depends  upon  the  mean  value  of  the  state  and  the 
applied  inputs.  Therefore,  different  residual  signals  may  be  optimum 
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for  different  mean  valures  and  inputs.  While  this  may  be  reasonable 
in  some  cases,  there  are  also  other  problems  in  which  it  is  possible 
and  in  fact  advantageous  to  find  a  single  set  of  residual  signals 
that  performs  satisfactorily  independent  of  operating  conditions, 
our  proposed  research  —  the  failure  projection  method  (FPM)  is  aimed 
at  this  problem.  In  this  thesis,  two  groups  of  formulations  are  deve¬ 
loped  to  solve  this  problem.  One  of  them  has  an  appealing  and  impor¬ 
tant  geometrical  interpretation  while  the  other  is  based  on  more 
practical  assumptions  so  that  it  would  be  more  useful  in  practice. 
Within  each  group,  several  formulations  involving  calculations  of 
decreasing  complexity  and  starting  from  slightly  different  criteria 
are  given.  We  develop  in  some  detail  the  relationships  (i.e.  the 
similarities  and  differences)  among  these  methods  so  that  the  issues 
involved  in  choosing  the  appropriate  formulations  for  a  given  problem 
are  evident. 

1.3  Goal  of  this  Thesis 

The  ultimate  goal  of  this  research  topics  is  to  find  a  vector  of 
residuals  g  such  that  g  is  small  when  no  failure  has  ocurred  and  has 
decidedly  different  characteristics  under  each  of  a  specified  set 
of  failures.  In  this  thesis  we  take  a  major  step  forward  towards  the 
achievement  of  this  goal  and  towards  the  development  of  a  deeper 
understanding  of  the  problem  of  robust  failure  detection.  We  do  this 


by  examining  the  problem: 


(1.2)  Minimize  the  maximum  residuals  of  the  normal  (unfailed)  system 
under  model  uncertainties. 

Note  that  this  problem  does  not  take  any  specific  failure  mode 
into  account,  but  rather  is  more  in  keeping  with  the  failure  detection 
philosophy  of  producing  as  large  a  set  of  signals  as  possible  which 
are  all  small  when  no  failure  has  ocurred.  The  implicit  assumption 
here  is  that  any  observable  failure  mode  will  lead  to  a  large  value 
for  the  residual  vector.  This  implicit  desire  can  be  made  explicit  by 
considering  problem  such  as 

(1.3)  Find  a  residual  vector  which  yields  good  performance  when  there 
is  only  one  postulated  failure  mode. 

Here  we  are  given  one  specific  failure  mode  to  detect  and  must 
find  a  residual  that  achieves  an  acceptable  tradeoff  between  detection 
and  false  alarm  characteristics.  A  variety  of  criteria  could  be  used 
to  determine  an  acceptable  residual  design.  For  example,  one  might 
constrain  the  size  of  the  norm  of  the  failure  residuals  when  the 
system  is  unfailed  while  maximizing  the  failure  residuals  when  the 
system  fails.  Clearly  one  can  consider  more  complex  problems  in  which 
there  are  several  failure  modes  postulated  and  we  want  the  residuals 
to  be  large  and  distinctly  different  for  each  (assuming  that 
we  wish  to  distinguish  among  the  different  failure  modes)  while  still 


requiring  them  to  be  small  under  normal  conditions. 


In  this  thesis  we  focus  essentially  all  of  our  attention  on 
Problem  (1.2)  and  in  particular  on  the  solution  of  this  problem  in 
the  presence  of  model  uncertainties.  The  frame  work  and  insight  that 
we  develop  also  provide  a  useful  starting  point  and  the  requisite 
machinery  for  considering  problems  such  as  (1.3)  which  incorperate 
specific  failure  mode  information.  We  will  commemt  on  such  extensions 
at  the  end  of  the  thesis. 

In  Chapter  2  we  introduce  the  basic  idea  behind  the  Failure 
Projection  Method  which  is  a  geometrical  approach  using  orthogonal 
projections.  Using  these  ideas  we  develop  a  first  group  of  basic  for¬ 
mulation  to  solve  Problem  (1.2).  Three  different  formulations, 
which  are  based  on  slightly  different  criteria  and  have  decreasing 
complexety  of  calculation  are  developed.  The  appealing  feature  of  this 
group  of  formulations  is  their  distinct  geometrical  interpretation  and 
the  intuition  they  provide  for  robust  failure  detection.  In  Chapter  3, 
another  group  of  formulations  is  proposed.  This  group  is  based  on 
assuming  that  one  has  available  a  priori  information  on  the  system 
state.  Specifically,  it  is  assumed  that  one  knows  that  the  state  is 
confined  to  a  given  ellepsoid.  Typically  in  practice  one  has  information 
about  the  likely  ranges  of  values  for  the  state  variables  and  conse¬ 
quently  this  group  of  approaches  may  be  more  useful  in  practice  than 
that  developed  in  Chapter  2 ,  which  uses  no  such  information.  In  Chapter 
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4  an  example  of  a  power  system  with  three  coupled  generators  is 
studied  in  order  to  show  how  the  formulation  obtained  in  Chapter  3 


using  singular  value  decomposition  can  be  applied  to  a  practical 
problem. 

One  of  the  key  aspects  of  analytic  methods  of  failure  detection 
is  the  use  of  information  concerning  the  dynamic  relationship  among 

outputs.  Consequently,  in  our  geometric  approach  we  consider  residuals 

generated  from  a  window  of  observations  over  some  interval  of  time. 

The  "observation  space"  then  has  dimension  proportional  to  the  length 

of  this  interval,  and  at  its  simplest  level  failure  detection  can  be 

achieved  by  projecting  the  observations  onto  the  orthogonal  complement 

of  the  subspace  in  which  the  observations  should  lie  if  there  is  no 

failure.  This  idea  in  fact  is  the  basis  for  our  geometric  approach 

in  the  next  chapter  for  the  problem  of  robust  failure  detection.  In 

addition  to  providing  the  motivation  for  our  robustness  analysis, 

this  idea  raises  an  additional  question.  Specifically,  if  we  are  given 

a  linear,  time-invariant  system,  how  do  we  determine  the  required 

length  of  the  window  of  observations  and  also  how  do  we  generate  the 

projection  onto  the  orthogonal  complement  just  mentioned.  In  Chapter 

5  we  present  a  frequency  domain  approach  to  determining  the  minimum 

window  size  and  for  generating  this  orthogonal  projection. 

Finally  in  Chapter  6,  we  shall  briefly  discuss  the  significance 
of  our  results  and  problems  which  should  be  examined  in  the  future, 
including  the  extension  of  our  methodology  to  problem  such  as  (1.3). 
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CHAPTER  2 

FAILURE  PROJECTION  METHOD  (FPM) 
—  GEOMETRICAL  APPROACH 


2.1  Introduction. 

In  this  chapter  we  introduce  the  basic  idea  of  the  FPM  and 
develop  three  formulations  which  have  geometrical  interpretations 
based  on  the  notion  of  the  angle  between  subspaces.  They  involve 
caculations  of  decreasing  complexity  —  from  nonlinear  programming 
to  singular  value  decomposition. While  having  appealing  geometrical 
interpretations  they  also  have  significant  limitations  which  will  be 
overcome  in  Chapter  3. 

To  introduce  the  basic  idea  of  the  FPM,  let  us  consider  a 
discrete  time  and  time  invariant  system 

(2.1a)  x(k+l)  =  Ax(k)  +  Bu(k) 

(2.1b)  y(k)  =  Cx(k)  k  =  0,1, . 

As  the  first  step  we  shall  focus  on  sensor  and  plant  failure 
detection  and  not  actuator  failures.  Thus  without  loss  of  generality, 

we  shall  assume  u(k)=0,  k=0,l, .  Therefore  the  system  equations 

we  are  dealing  with  can  be  written  as 

(2.2a)  x(k+l)  =  Ax (k) 

(2.2b)  y (k)  =  Cx (k)  k=0,l, . 


12 


where  ::(k)  e  P.n,  y(k)  e  Rm,  A  e  R0*" 
Define  the  extended  observation 


and  C  e  R1*". 

vector  of  length  p  to  be 


(2.3) 


yp(k)  = 


ryOO 

y(k+D 


Ly(k+p-l). 


P » k=0 , 1 , 


Obviously 


(2.4) 


yp(k) 


x(k) 


Let  s=mp.  Then  yp  (k)  e  RS. 
Define 


(2.5) 


C 

CA 


e  R 


s*n 


CA 


P-1 


and  the  observation  apace  (of  length  p)  Z  to  be 

P 

z_  =  R(C  ) 

P  P 


(2.6) 
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where  R(C^)  is  the  range  of  .  Define  the  detection  space  (of  length 
p)  to  be  the  orthogonal  complement  of  2 i.e. 

(2.7)  Z  ©  G  =  rs 

p  p 

Z  G 

P  P 

Then  for  any  vector  g  eG  we  have 

P 

(2.8)  <g,yp>  =  0 

Equation  (2.8)  (or, more  precisely,  the  left-hand  side  of  this  equation) 
is  called  a  parity  check  [2] , [4] .  We  shall  call  g  a  detection  vector 
(of  length  p) . 

In  (2.8),  if  p=l,  then  y^  =y(k) .  Therefore  we  only  make  use  of 

the  redundancy  among  values  of  the  measurements  at  a  single  time, 
as  considered  by  Potter  and  Suman  in  [4]  and  Desai  and  Ray  in  [5] . 

If  p>l,  we  can  make  use  of  the  redundancy  among  measurements  at  diffe¬ 
rent  time,  as  discussed  at  some  length  by  Chow  [2]  and  Chow  and 
Willsky  (3] . 

A  parametric  failure  is  one  that  causes  parameter  changes,  e.g. 
changes  in  A  and/or  C.  An  additive  failure  is  one  that  creates  additive 
components  in  (2.2a)  and/or  in  (2.2b).  For  example, 

(2.9)  x (k+1)  =  Ax  (k) 


y  (k)  =Cx  (k)  +  f  (k) 
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where  f (k)  represents  some  additive  failure.  Either  of  these  failures 
will  generally  result  in  a  nonzero  value  for  at  least  one  parity 
check.  Thus  a  set  of  parity  checks  serves  as  a  failure  detector 
which  ideally  is  zero  if  there  is  no  failure  and  is  non-zero  if  a 
parametric  failure  or  an  additive  failure  has  occurred. 

Motivated  by  these  ideas,  Potter  and  Suman  [4]  proposed  a  new 
approach  which  makes  use  of  a  complete  set  of  detection  vectors. 

In  this  approach  a  matrix  V  whose  rows  are  an  orthogonal  basis  for  the 
detection  space  G^  (i.e.  a  complete  set  of  detection  vectors)  is 
defined.  Because  G  is  the  orthogonal  complement  of  Z 


vy„  -  0, 


any  y?e 


The  range  of  V,  R(v),  is  called  the  parity  space  [4] , [51 .  The  vectors 

which  belong  to  the  parity  space  are  called  the  parity  vectors. 

As  mentioned  before,  in  [4]  and  [5]  the  special  case  p=l  is 

considered.  Here  we  use  the  definition  of  extended  observation  vector 

y  for  p>l  introduced  by  Chow  in  [2] . 

P 

In  this  chapter  we  will  make  a  particular  choice  for  V.  We 
define  V  to  be  the  orthogonal  projection  along  onto  G^  denoted  by 
P  .  Note  that  P_  is  an  operator  from  RS  to  RS,  while  in  [4]  V  is  an 
operator  from  Rs  to  R^  where  dimG  =q.  Note  also  that  rank  PG=q. 
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It  is  known  that  an  orthogonal  projection  has  some  important 
properties,  i.e. 


(2.!°)  PG  =  PG  =  V 

(2.11)  R (P_)  =  G  N( P  )  =  2 

V3  P  G  P 

(2.12)  PG  =  G<G,Gf]G' 


where  G  is  a  matrix  the  column  of  which  form  a  basis  for  subspace  G^. 
We  shall  see  later  on  that  the  geometric  and  algebraic  properties  of 
the  orthogonal  projection  enable  us  to  pose  problems  involving  un¬ 
certainties  relatively  easily.  This  is  our  reason  for  using  it. 


2.2  FPM  With  Uncertainties  —  Problem  (1.2) 

To  characterize  the  influence  of  uncertainties  let  us  introduce 
a  compact  set  of  possible  parameter  values  U  (e.g.  a  subset  of  a 
space  for  some  t)  and  two  functions  A  and  C 


A  : 

C  : 


U  R 


nw 


U  -*  R 


m>ci 


Then  we  can  incorparate  parameter  uncertainties  in  the  system  equa¬ 
tions  (2,2a)  and  (2.2b)  by  modifying  them  as  follows: 


6 


(2.13a)  x(k+l)  =  A(n)x(k) 

(2.13b)  y(k)  =  C(h)x(k) 


k=0,l, . .  Tie  U 

Here  A(p)  and  C(ri)  represent  the  possible  system  matrices  as  r)  varies 
over  U.  They  could  have  different  ranks  for  different  peu  due  to 
unmodeled  dynamics.  Similarly 


(2.14) 


yp(k)  - 


c(n) 

c(n)A(n) 

% 

* 

c(n)A(n)p_1 


x  (k)  =  C  (T) )  x  (k) 

P 


2  (ri)  =  r(c  (n)) ,  ne  u 

p  p 


It  can  be  seen  that  as  n  varies  over  U,  Z  (n)  will  vary  also 

P 

(in  fact,  in  general  its  dimension  will  change  as  h  is  varied) . 


Generally  we  cannot  find  a  G  which  is  orthogonal  to  all  Z  (n)  as 

P  P 

we  did  in  the  previous  section,  that  is  to  say  we  cannot  find  a  P 


which  projects  all  vectors  in  Z  (n)  (for  all  p  )  to  zero,  In  order  to 

P 

formulate  Problem (1.2)  precisely  what  we  must  intuitively  seek  is  a 

P_  which  makes  the  projection  of  vectors  in  Z  (T|)  as  small  as  possible 
G  p 

for  all  n  .  To  obtain  some  insight  into  this  problem  let  us  consider 
a  simple  example  where  U={l,2,3}  and  Z(1),Z  (2)  and  Z (3)  are 
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one-dimensional  subspace  of  a  three-dimensional  space.  Then  one 
natural  choice  for  the  detection  space  G  is  that  which  minimizes  the 
maximum  projection  of  unit  vectors  in  Z(1),Z(2)  and  Z(3).  That  is  , 
this  choiceminimizes  the  worst  possible  effect  of  parameter  uncertain¬ 
ties.  The  nature  of  this  approach  can  be  visualized  as  in  Fug.  2.1 
where  s3,s2  and  s3  are  projections  of  unit  vector  in  Z(1),Z(2)  and 
Z(3)  onto  a  candidate  2-dimensional  space.  Our  goal  is  to  find  a  G 
such  that  the  maximum  vector  among  s3,s2  and  s^is  minimized. 


Fig.  2.1  Failure  projection  with  system 
uncertainties 


Generally 


Z 

P 


(h) ,  peU  may  have  different  dimensions.  Suppose 


r  =  max  d  ( Z  (r>) ) 

neu 


where  d(Z(p))  is  the  dimension  of  Z  (n) .  Then  our  formulation  of 


Problem  (1.2)  would  be 
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(2.15) 


2 

inf  sup  sup  ||  P  y  || 

G  neu  yeZ  (ri ) 

dim  G=s-r  |y|f=l 


In  the  next  section  we  shall  further  discuss  Problem  (2.15)  and 
eventually  find  a  way  to  solve  it  in  terms  of  a  basis  of  Z(H).  Fur¬ 
thermore,  in  siction  2.4  and  2.5  we  shall  develop  other  alternative 
formulations  to  problem  (1 ,i\  which  turn  out  to  be  much  easier  to 
solve  and  also  possess  advan  .i  it  s  in  practice  over  the  approach 
discussed  in  this  and  the  following  section. 


2.3  Solution  to  Problem  (2.15) 

In  this  section  we  shall  continue  the  discussion  started  in  section 
2.2  in  order  to  understand  more  thoroughly  the  problem  we  have  described 
and  to  develop  a  solution  in  terms  of  a  basis  of  Z(n). 

Consider  Eq. (2.15).  Let  us  suppose  U  is  a  finite  set,  e.g. 

(2.16)  U  =  (1,2, . ,t) 

We  make  this  assumption  for  mathematical  convenience.  However,  in 
principle  one  can  discretize  the  set  of  possible  parameters.  Fur¬ 
thermore,  it  is  an  unproven  conjecture  that  any  set  U  can  be  replaced 
by  a  finite  set  whose  corresponding  observation  subspaces  are  the 
"extreme  points"  of  the  original  set  of  observation  spaces  as  the 
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parameter  vector  ranges  over  U. 

Under  the  assumption  (2.16),  problem  (2.15)  can  be  rewritten  as 

2 

inf  sup  sup  |]p  y ||  ,  i=l, . ,t 

G  ieu  yez. 

IHK 

Suppose  d(Z  )=nu.  Let  Z^  and  G  be  matrices  whose  columns  are 
orthonormal  bases  for  and  G  respectively.  Then  according  to  (2.12) 
we  have 

(2.17)  PG  =  G(G'G)_1G*  =  GG' 

and 

sup ||  PGy||  2  =  sup ||  GG'y ||  2 
ye  Z.  ye  Z. 

lUhi1 

Let  y=Z.x,  xeRm  .  Then 

l 

sup  ||  PGy|j  =  sup ||  GG'Z.x|| 

YGZi  l^^lh1 

IMb1  xERm 

l^xlf-  x'Z.’Z.x  -|Hf 


But 
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So 


sup  II  P  y  ||2  =  sup  xlZ.  '■GG'GG'Z.x  =  sup  x'Z.  ’GG'Z.x 

y<  0  II  *II-i  4  1  IMM  1  1 

||y||=l  xeRm  xeRm 


=o2  (G'Z.)  =  a2  (Z.  'G) 

max  i  max  i 


where  cr^.)  is  the  maximum  singular  value  of  a  matrix.  The  last 

equality  holds  simply  because  0  (A)=<T  (A')  for  any  matrix  A. 

iHftX  rndx 

Therefore  we  have 


(2.18) 

Combining 
in  section  2,2 
normal  columns 


sup  P  V  2=  o  2  G’Z.)  =  o  2  Z  G) 

_  G  "  max  l  max  i 
y£  ti 

llylhl 

(2,16)  and  (2.18)  we  have  that  the  FPM  problem 
reduces  to  solving  for  a  nx(s-r)  matrix  G  with 
and  which  achieves 


posed 

ortho- 


(2.19) 


min  max  a  (G'Z.) 

.  max  i 

G'G=I  1 
s-r 


The  problem  given  in  Eq. (2,19)  has  a  simple  geometrical  inter¬ 
pretation.  If  we  define  the  angle  between  two  sets  s,  and  s  ,  s.  s  £Rn 

to  be 

A(s]l,s2)  =  sup  ||  Pg  x  ||  2 
xts^^  2 

11x11*1 


(2.20) 


where  Pg  x  is  the  orthogonal  projection  of  x  on  s2,  then  formulation 
(2.19)  can  be  given  a  geometrical  explanation.  To  see  the  meaning  of 
definition  (2.20)  let  us  consider  a  2-dimensional  case.  As  in  Fig. 2. 2, 
s^  and  s^  are  two  subspaces.  The  cosine  of  the  angle  between  and 


s2  can  be  seen  to  be  the  orthogonal  projection  of  unit  vector  of 
on  s2  as  shown  in  Fig.  2.2. 

Using  definition  (2.20) ,  (2.19)  can  be  rewritten  as 

(2.21)  min  max  A(Z  .  »G) 

G'G=I  i  1 

s-r 

Eg.  (2.21)  means  that  the  optimal  G  should  be  that  which  minimizes 
the  maximum  angle  between  and  G  .  This  is  the  geometrical  meaning 
of  formulation  (2.19). 


The  solution  of  (2.19)  requires  the  use  of  nonlinear  programming. 
We  have  chosen  not  to  do  this,  since  as  we  will  discuss  in  Chapter  3, 
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the  mathematical  formulation  on  which  (2.19)  is  based  has  some  weaknesses 
in  terms  of  its  relationship  to  the  nature  of  real  failure  detection 
problems.  As  a  prelude  to  the  discussion  in  Chater  3,  let  us  consider 
several  alternatives  to  (2,19)  which  are  less  difficult  to  solve, 
which  in  some  sense  can  be  viewed  as  approximations  to  (2.19),  and 
which  in  others  can  be  interpreted  as  solving  slightly  different 
problems  that  possess  advantages  over  our  original  formulation. 

As  a  first  step,  it  is  easily  shown  that 


t 

E 

i=l 


max 


(G'Z.) 

l 


t 


2  2 

<  max  a  (G'Z.)<  E  a  (G'Z.) 
“  .  max  l  .  ,  max  l 

l  1=1 


Therefore  instead  of  solving  (2.19)  one  might  consider  solving  the 
following  problem 

t 

E  a  2  . 

max  (G'Z.) 

i  i 


(2.22)  min 

G'G=I 


which  gives  upper  and  lower  bounds  to  (2.20)  because  if  G^  is  the 
solution  to  (2.22)  and  G^  is  the  solution  to  (2.19),  then  we  have 


t 

E 

i-1 


a  (G'z.) 
max  1  i 


t 

E 

i-1 


a  (Glz.) 

max  2  i 


<  max 


o  (g;z.) 

max  2  i 
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<  max  a  2  (G  'z.)<  E  a  2  (G ,'Z.) 

.  max  1  l.  ,  max  1  i 
1  2  =  1 


or 

t 

(2.23)  min  E  a  (G'Z.) 

G  i=l  m3X  1  .  2 

- —  <  mm  max  CT  (G'Z.) 

—  _  .  max  i 

t  G  l 

<  min  E  0  2 (G ' Z . ) 

~  .  ,  max  l 

G  1=1 

It  should  be  pointed  out  that  in  addition  to  its  interpretation 

as  an  approximation  to  (2.19),  the  formulation  (2.22)  has  its  own 

physical  interpretation.  Suppose  we  know  the  a  priori  probabilities 

of  occurance  of  different  possible  uncertanties .  Then  (2.19)  might 

not  be  reasonable  because  the  occurance  of  the  uncertanty  which 

gives  maximum  projection  may  be  very  unlikely.  A  more  reasonable  solution 

would  be  the  minimization  of  the  weighted  summation  of  the  maximum 
2 

projections  0^^  (G'ZJ.  That  is  if  p^,  i=l,....,t  are  the  probabili¬ 
ties  of  the  occurance  of  different  uncertainties,  then  we  would  like 
to  minimize  the  weighted  summation: 


(2.24) 


min 

G'G=I 


£p‘ 


2 


0 

max 


(G'Z  ) , 


t 

E 

i=l 


V1 


Geometrically  here  we  minimize  the  expectation  of  maximum  projection 
instead  of  minimizing  the  maximum  among  them. 


Nl 


If  the  uncertainties  are  equally  likely  to  occur,  i.e. 

p^=p^=  . =pt  then  we  have  (2.22).  That  is  to  say  (2.22)  is  only 

a  special  case  of  (2.24).  On  the  other  hand,  if  we  define  new  matrices 
.  to  be 

l 


/p\  Z . 

l  i 


i=l» 


then  (2.24)  can  be  rewitten  as 


(2.22a) 


min 


t 

Z  a 


G'G=I  i=l 


(G ' Z  . ) 
max  i 


Therefore (2 . 22)  can  be  thought  of  as  a  normalized  version  of  (2.24). 

Because  of  inequality  (2.23),  it  is  at  least  plausible  that  in 

most  problems  the  G  that  solves  (2.22)  will  be  a  useful  approximation 

to  the  G  that  solves  (2.19).  (We  will  provide  more  insight  into  tM * 

t  2 

point  through  example  in  Section  3.3).  Also  because  Z  0max  (G’Z^) 

i=l 

is  differentiable  with  respect  to  g. .  (the  entries  of  G) ,  the  problem 

13 

(2.22)  can  be  solved  more  easily  than  (2.19)  using  nonlinear  programming. 

Although  problem  (2.19)  and  (2.22)  can  be  solved  in  principle 
by  nonlinear  programming  we  would  like  to  have  an  alternative  approach 
which  gives  essentially  a  closed-form  solution.  Such  an  approach 
is  developed  in  the  next  section,  and  as  we  will  see,  it  also  gives 
additional  insights  into  the  problem  of  failure  detection  in  the 


presence  of  uncertain  parameters . 
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2,4  An  Alternative  Approach 

In  this  section  we  develop  an  alternative  approach  which  can  be 
thought  of  as  giving  an  approximate  solution  to  (2.20)  and  (2.22) 
and  can  be  solved  simply  using  a  singular  value  decomposition.  To 
begin,  it  is  known  that 


a  (G'Z.)  = 
max  l 


G'zJlj  =11  VGll2 


where  j|*  ||  is  the  induced  matrix  norm.  Thus  (2.22)  is  equivalent  to 


(2.25) 


min  Z  ||  Z  * G U  , 
G  '0=1  i=l  1 


The  Frobenius  norm  of  matrix  A e  CWn  j  s  defined  to  be 


Mf  ' r  E  Kj  I2 

1  3 


Because  I' A’’  is  the  trace  of  A'A 


Ml 


i  1 


tr(A'A)  =  Z  a. 

i-1  1 


where  is  the  eigenvalues  of  A'A  and 


A ||  2  =  max 

i  J' 
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l 

I 

‘  t 

t 

l 


It  can  be  easily  shown  that 


Therefore  using  the  similar  arguments  given  in  proving  (2,23) ,  we  have 


(2.26) 


min  £  ||  Z'g|£ 
G  i=l 


s-r 


< 


t 

min  £ 

G  i=l 


Z. 

l 


min  £  ||z  ’g|£ 
G  i=l 


Following  the  same  line  of  reasoning  as  that  used  in  the  preceding 
section,  we  are  led  to  the  following  problem. 

(2.27)  min  £  ||  Z.  'Gjf 

G ' G=I  i=l  1 

As  mentioned  before,  we  may  use  Z,=/p, Z.  instead  of  Z.  in  (2.27)  if 

ill  l 

the  probabilities  of  occurance  of  uncertainties  p^  are  different  from 
each  other. 

The  problem  (2.27)  can  further  be  written  as 


(2.28^ 


min  |{Z'G|P 
G'G=I 


where 


(2.29) 


2? 


In  the  remainders  of  this  section  we  discuss  two  very  important 
points.  The  first  is  that  (2,28)  can  be  thought  of  as  a  different 
but  fundamental  approach  to  solving  the  problem  of  generating  robust 
parity  checks  for  failure  detection.  The  second  is  that  this  problem 
can  be  solved  very  easily. 

Now  we  derive  the  solution  to  problem  (2.28),  Suppose  the  singular 
values  of  Z  are 

(2.30)  a.  <  cr< . <a 

12  -  s 


We  claim  that 


(2.31) 


min  l|z'G||p  =  £  O  .2 

G'G=I  i=l 


and  the  optimal  G=G*  is  given  by 

(2.32)  G*  =  [  g, . g  ] 

1  s-r 

where  g^ . . ,gs-r  are  tlle  s^n9ular  vectors  of  Z  corresponding  to 

0^, . ,0r  respectively. 

To  justify  our  claim  we  first  prove  the  following  lemma. 

Lemma  2 . 1 
If 
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is  symmetric  and  positive  semidef inite,  and  A^  is  m  by  m,  A  is  n 
by  n,  and  eigenvalures  are  written  in  increasing  order,  then 


(2.33) 

X. 

l 

(A)  <  Xi(A11) 

m 

m 

(2.34) 

E 

X .  (A)  <  E 

1  i=l 

1=1 

1<  i<m 


'll 


Proof : 

According  to  the  Courant-Fischer  minimax  principle  [10] ,  we  have 


(2.35) 


X j (A11)  *  min  max 


X’A11X 


P,  X€P.  X*X 

j  3 


where  P_.  is  a  subspace  of  dimension  j,  the  orthonormal  basis  of  which 
is  p^, . ,Pj  Now  let  us  define 


i=lr . 'j 


where  and  are  m  and  n  vectors  respectively.  Then  (2.35)  can  be 
rewritten  as 


(2.36) 


Xj  (An) 


min 


max 

yeQ. 


^lly 

y’y 


where  Q ^  is  the  subspace  spanned  by  ....... ,q^ . 


Also  from  the  Courant-Fischer  minimax  principle , 


(2.37) 


X  .  (A)  =  min  max  —  —  -JSL, 
1  F  VEF  y'y 


where  F  is  any  subspace  of  dimension  j.  Comparing  (2.36)  with  (2.37) 
it  is  not  difficult  to  see  that 


X  (A)  <  X.(An)  ,  j=l . k 


Now  let  H  be  any  matrix  satisfying 


H'H  =  I 


,  is  s-r  by  s 


H'ZZ'H 


H^ZZ’H1  * 


HU  * 


z'Hlil  F  =  trace  <Hn) 


According  to  Lemma  2.1,  (2.34) 


s-r  s-r 

E  X.(ZZ')  =  I  X,  (H'ZZ’H)  <  trace(H,,)=||  Z’H, 
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for  any  H  satisfying  The  equality  holds  when  the  rows  of 

H‘  are  the  eigenvectors  of  ZZ *  corresponding  to  its  smallest 
eignvalues.  Since 

X  i  (ZZ 1 )  =  a\ 

we  complete  the  proof  of  (2.31)  and  (2.32). 

Therefore  (2.31)  and  (2,32)  give  a  very  simple  solution  which 
is  directly  given  via  a  singular  value  decomposition  of  Z,  and  thus 
easier  to  be  solved  than  the  original  problem  (2.19)  or  (2.22).  It 
should  also  be  pointed  out  that  (2.31)  and  (2.32)  have  other 
intuitive  interpretations  which  are  given  in  the  Appendix  to  this 
chapter . 


2.5  FPM  With  Noise  And  Uncertainties 

So  far  we  have  discussed  the  FPM  with  modelling  uncertainties 
only.  Introducing  process  and  measurement  noise  requires  some 
adjusment  of  our  approach,  as  the  observations  y(k)  are  no  longer 
deterministic  but  random  vectors.  In  this  case  a  good  FPM  must 
minimize  both  the  mean  value  and  the  variance  of  the  maximum  norm 
of  failure  projection  if  no  failures  occurred.  The  system  equations 
are 


x(k+l)  -  A(T))x(k)  +  D(n)w(k) 
y(k)  *  C(p)x(k)  +  v(k) 
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where  w(k)  and  v(k)  are  driving  noise  and  observation  noise  respecti- 


vely  and 

(2.39) 

E [w (k) ]  =  0,  E[v(k) )=0 

(2.40) 

E[w(k)w' (t) ]=Q(k)6kt  , 

(2.41) 

E(v(k)v* (t))=R(k)5kt  , 

kt 


1,  k=t 

0,  Mt 


k, t=l f 2 , 


The  extended  observation  vector  y(k)  can  be  expressed  as 


(2.42) 


y  (k)  = 


C 

CA 


CA 


P-1 


x  (k)  + 


y(k) 

y (k+p-l) 


0 

0 

CD 

0 

CAD 

CD 

• 

• 

0 

CAP“2 

CAP'3 

CD 

0 

w(k) 


Lw  (k+p-l  )J 


v  (k ) 


L  V  (k+p-l  )J 


Define 


C  = 


C 

CA 

Lca1 


p-1 


Therefore 

(2.43) 

Prom  (2.39) , 
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y (k)  =  Cx (k)  +  Dw(k)  +  v (k) 
.40)  and  (2.41) 


E[w(k) ]  =  0  ,  E  [v  (k) )  =  0 


0(lt) 


E(w(k)wMk) )  = 


=  Q(k) 


0(10  J 


F  (k) 


E[v(k)v' (k)  1  = 


=  R(k) 


R(k) 
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Recall  the  deterministic  case  in  section  2.2  where  the  quantity 
we  would  like  to  minimize  is  (see  (2.15)) 


(2.44)  I  =  max  max  j|  P^lP 

neu  y eZ  on) 
fly  ll=i 

In  this  present  case,  because  the  observation  vector  y  is  no  longer 

a  deterministic  vector,  instead  of  |'j  P  y||  we  would  like  to  use 

G 

2 

its  expected  value  Ejjp  y]|  .  From  (2.43)  we  see  that 

G 

Ey  =  C  (Ex) 


So  that 


Ey  ER(C)  =  Z 


Therefore  a  natural  counterpart  of  (2.44)  in  stochastic  case  is 


Or 

(2.45) 


I 


2 

max  max  E  HP  y(h)  H 
neu  EyeZ  (n ) 

IferIM 


I  =  max  max  E ( (P_y) ' (P_y) ] 
.  G  G 

n  zcZ^n) 


=  max  max  tr[E[(P  v) (P„y) ']] 

n  zeZ(n)  G 


where 

(2.46)  z(n)  =  E (y)  =  C(h)x 

Define 


P  =  GG' 


34 


Prom  (2.17) f 


(2.47) 


Consider  the 


Therefore  (2 


(2.48) 


2.43)  and  (2,46)  we  have 

e[ (PGy(n)>  (PGy(n>) '] 

=  F(p(z(n)+6(n)w  +  v) (z(n)  +  6(n)w  +  v)'p'] 
=E(Pz(n)2’ (n)p’  +  p6(n)ou'(n)  p’ +  prp’j 

trace  of  (2.47) 

T(n)  =  tr[pz(n)z' (n)p’  +  PD(n)pD' (n)p'  +  prp'] 

=  (pz(n)) ' (P2(n))  +  tr[p6(n)o6’ (h)p'l  +  trfPRP'] 
=  (Pz(ti))  ’  (Pz(r)))  +  tr[5(TDpff(n)P'P]  +  tr [RP'P] 
=(Pz(h) ) ‘ (Pz(h) )  +  tr[D(h)QD' (h)P)  +  tr[RP] 


.45)  can  be  written  as 


1  =  max  max  T(n) 

n  zeZ(n) 

=  max  max  {(Pz  ( n) )  '  (Pz  ( h) )  +  tr[M(n)P]+  tr(RP)  ) 

n  zeZ(n) 

=  max  {  max  [ (Pz) ’ (Pz) ]  +  tr [M ( n)P) )  +  tr (RP) 

n  zeZ(n) 

=  max  {o  (Z’(n)G)  +  tr[M(n)GG'l  )  +  tr(RGG') 
max 

n 
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where 


M(n)  =  5(n)QD'  (n) 


Therefore  Problem  (1.2)  becomes 


(2.49)  min  max  T(h) 

G '  G=I  ze  (n) 

=  min  {max  [  0  2(Z'(ri)G)  +  tr (M(n)GG' ) ]+tr (RGG' ) 

.  max 

G  G=I  T) 

If  n  belongs  to  a  finite  set,  as  we  considered  in  Chapter  2,  Section 
2.3,  (2.49)  can  be  written  as 


(2.50) 


min 

G'G=I 


(  max (0 


(Z !G) +tr (M.GG ’ ) ] +tr (RGG ' ) ) 

1  l 


It  is  obvious  that  (2.50)  can  be  solved  by  nonlinear  programming . 

Using  the  same  arguments  we  used  in  deriving  (2.22)  or  (2.24), 
we  arrive  at  an  alternative  criterian 


(2.51) 


min 

G'G=I 


t 

E  [  a 

i=l 


max 


2 


(Z ! G) +tr (M . GG ' ) ]  +  tr(RGG')) 
1  1 


min 

G'G=I 


2  — 

I  a  (Z!G)  +  tr[(  E  M . ) GG ' ) +tr (RGG  * ) } 
.  ,  max  i  .  ,  l 

i=l  1=1 


Finally  we  have 
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(2,52) 


min 

G'G=I 


t 

{  l  a 


i=l 


max 


2 


(Z!G)  +  tr(NGG') 

l 


} 


where 


t 

N  =  l  M.  +  R 

i-i  1 


It  is  easy  to  see  that  (2,50)  and  (2.52)  are  counterparts  of  (Al) 
and  (A2)  respectively.  Similarly,  we  have  the  counterpart  of  (A3), 


(2.53) 


min  {  I  ||  z;g|^  +  tr(NGG')  ) 
G'G=I  i=l  1 


Because  N  can  be  partitioned  as 
N  =  SS  * 

Then  (2.53)  can  further  be  simplified  as 


min  (  llz-Glg  *  lls'clg  1 

G  G=  X 


Or 

(2.54)  min  {||z'g|£} 

G’G=I  F 


Z'  = 


where 
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Therefore  our  problem  has  been  reduced  to  the  same  form  as  (2.28) 
and  can  be  solved  using  (2.31)  and  (2.32).  The  only  difference  is 
that  instead  of  Z  in  (2.31)  and  (2.32),  here  we  use  Z=[Z  S]  and 
S  is  a  matrix  related  to  the  covariance  matrices  of  noise  Q  and  R 
Because  (2.54)  can  be  written  as 

,  s_r 

(2.55)  min  ’iz-Gjr  =  T. 

G'G=I  T  i=l  1 

where  is  the  eigenvalues  of  ZZ‘  arranged  in  decreasing  order. 
And 

(2.56)  7.Z'  =  7.7.'  +  SR' 

Also  it  can  be  proven  that  110] 

(2.57)  X.(ZZ’)  =  X.(ZZ'+SS')  ^A.(ZZ’) 

li  l 

Combining  (2.55)  '2.57)  and  (2 . 28)  ,  (2 . 31 )  we  have 

(2.58)  min  i!z'G;£  £  min  |fc'G|£ 

G ' G= I  G 1 G-I 

What  (2.58)  tells  us  is  that  introducing  noise  increases  the 
projection  of  the  normal  system.  If  there  is  no  noise,  S=0,  then 
(2.55)  reduces  to  (2.28). 
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The  robust  failure  detection  issue  has  been  considered  in 
Chow's  thesis  [2]  where  a  relationship  between  sensor  outputs  is 
selected  so  that  the  rasultant  residual  will  be  as  small  as  possible 
under  worst  case  conditions  on  parameter  uncertainties  and  noise 
when  no  failure  has  occurred.  Chow  develops  a  minimax  design 
procedure  for  determining  the  best  residual  signal.  In  this  formu¬ 
lation  the  choice  of  residual  signal  depends  upon  the  mean  value  of 
the  state  and  the  applied  inputs.  Therefore .different  residual 
signals  may  be  optimum  for  different  mean  values  and  inputs.  Conse¬ 
quently,  one  can  imagine  a  system  in  which  different  residual  genera¬ 
tion  system  are  used  under  different  operating  conditions.  In  some 
problems,  then  the  operating  conditions  are  known  a  priori  or  in 
which  conditions  are  such  that  one  can  consider  scheduling  the 
residual  generation  procedure,  this  approaches  is  clearly  quite 
costly  in  terms  of  off-line  and  on-line  calculations  and  also  in 
many  problems  may  be  unnecessary  {  because  there  are  residuals  which 
work  well  under  all  conditions)  or  undesirable.  The  methods  outlined 
in  this  chapter  and  in  the  next  directly  address  the  problem  of 
finding  a  single  set  of  residual  signals  that  performs  satisfactorily 
independent  of  operating  conditions.  Our  formulation  (2.45)  can  be 
directly  interpreted  as  choosing  the  residual  process  under  worst 
case  conditions  as  opposed  to  under  a  specific  condition  as  is  done 
by  Chow,  While  this  formulation  requires  nonlinear  programming,  as 
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does  Chow's,  our  related  formulation  (2.54)  is  far  simpler  to  use  as 
it  only  involves  performing  a  singular  value  decomposition. 


2 . 6  Summary 

In  this  Chapter  we  have  developed  three  different  approaches, 
namely  (2 , 19) ,  (2 . 22)  and  (2.27)  and  their  corresponding  noise 
versions  (2 . 50) ,  (2 , 52 )  and  (2.54)  for  the  specification  of  robust 
parity  checks.  There  are  several  points  worth  noting  about  these. 
Because  of  the  similarity  between  the  two  groups,  we  focus  our 
comments  on  the  first  group  in  the  following  discussion.  The  results 
can  be  easily  applied  to  the  noisy  situation  as  well. 

The  first  concerns  numerical  solution.  The  three  problems  (2.19), 
(2.22)  and  (2.27)  involve  calculations  of  decreasing  complexity. 
Problem  (2.19)  and  (2.22)  both  require  nonlinear  programming,  but 
the  criterion  in  (2.19)  is  non-dif f erentiable  while  that  in  (2.22) 
is.  Problem (2. 27)  involves  only  a  singular  value  decomposition. 

The  second  point  is  that  the  three  criteria  have  intuitively 
appealing  geometric  and  algebraic  interpretations.  In  particular, 
(2.19)  solves  for  the  subspace  that  maximizes  the  minimum  angle 
between  the  subspace  and  all  of  the  possible  observation  spaces, 
while  (2.22),  which  can  be  viewed  as  an  approximation  to  (2.19), 
maximizes  the  expected  angle  between  the  subspace  and  the  obser- 
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observation  space,  there  the  expectation  is  over  the  possible  para¬ 
meter  values.  Finally  (2,27),  which  can  also  be  viewed  as  an  appro¬ 
ximation  to  (2.22),  has  its  own  physical  interpretations.  For  example 
it  can  be  interpreted  as  minimizing  the  volume  of  projection  of  the 
Z  on  G  (see  Appendix  of  this  Chapter) . 

The  third  point  is  that  these  formulations  have  significant 
limitations.  In  particular,  in  computing  the  angle  between  subspaces, 
one  effectively  examines  the  inner  product  of  all  possible  unit 
vectors  in  the  two  subspaces ,  An  implication  of  this  in  our  context 
is  that  all  directions  in  the  observation  spaces  are  equally  likely 
to  occur  with  unity  magnitude.  There  are  two  problems  with  this. 

The  first  is  that  in  the  generic  situation  parameter  variations 
may  increase  the  dimension  of  the  observation  space  by  introducing 
a  small  component  into  the  matrix  C  .  However  in  Z^  this  direction  is 
given  equal  footing.  Thus,  the  approach  outlined  here  may  produce 
too  few  parity  checks.  Secondly,  typically  one  has  a  significant 
amount  of  information  about  the  relative  sizes  of  the  state  x,  and 
clearly  this  information  translates  into  some  a  priori  information 
about  the  likely  relative  magnitudes  one  would  expect  to  see  in 
different  directions  in  the  observation  space.  This  information  is 
completely  ignored  in  the  approach  described  in  this  Chapter.  In  the 
next  Chapter  we  sacrifice  some  of  the  geometric  interpretation  to 
develop  three  approaches  that  parallel  (2,19),  (2.22)  and  (2.27) 


but  that  use  the  available  information  about  x. 


APPENDIX 


In  this  Appendix  we  explain  the  geometrical  meaning  of  formulation 
(2.28)  in  two  different  ways.  The  first  one  comes  from  looking  at 
the  problem  (2.15)  in  the  following  way.  First,  find  a  subspace  ZQ 
of  dimension  r  such  that  is  "closest"  to  Z^,  . . , . ,Zfc  ,  where 
"closest"  means  to  choose  a  Z^  such  that 


min  || 
d(ZQ)=r 


here  Z= [Z^ . Zfc]  as  defined  before  and  ZQ  is  a  matrix  whose  columns 

span  the  subspace  ZQ. 

Second,  find  the  orthogonal  complement  of  ZQ  as  our  G.  This 
can  be  seen  intuitively  from  Fig.Al ,  This  G  is  the  same  as  that 
which  results  from  the  solution  of  problem  (2.28). 


To  prove  this  result,  let  us  first  introduce  the  following  lemma. 


rv 


Lenina 
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Let  Z  be  mxn  real  matrix.  Its  singular  value  decomposition  is 
Z  =  Ul  V 


where 


Z  = 


*  .  0 


(if  n^m) 


* '  *  ’ ain  are  s:*-n9u^ar  values  of  Z.  Let  r  be  an  integer  with  r<m. 
Then  the  solution  to  the  following  problem 


min  {||Z-Z0!1M 

zo 

dim(Z  )=r 
o 


z0  =  u 


m-r+1  0 

•a 

m 


proof : 

see  [10] . 

0 

Let 

U  =  [u. . u  ] 

X  Itl 

If 

G  =  [u, . u 

1  m-r 

and 

“a 

m-r+l 

S  * 

• 

•  a 

Then 

VG  = 

1 1  ’"1 

o  o  o 

O  CO  o 

T  T 

i  G  =  V 


[o  s][c]  .0 


43 


As  we  see,  G  is  the  same  as  that  of  (2.28). 

The  other  interpretation  comes  from  the  problem  of  choosing  a 
G  such  that 


min  det(G'ZZ'G) 

G'G=I 

This  problem  also  has  an  interesting  interpretation.  Because 
det(G*ZZ’G)  can  be  thought  of  as  the  volume  of  the  projection  of 
the  vectors  in  Z  on  G  . 


The  solution  G  is  the  same  as  (2.28).  The  proof  is  similar  to 
that  of  (2.28),  because  if  we  define 

D  =  G'ZZ'G 


as  before,  then 


det (D) 


s-r 

n 

i=l 


s-r 

n 

i=l 


X 


i 


where  X^  and  X^are  eigenvalues  of  D  and  ZZ'  respectively.  The 
minimum  arrives  if 


G 


[u. 


u 

s-r 


) 


as  we  proved  before. 


CHAPTER  3 


GEOMETRICAL  APPROACH  WITH 
CONSTRAINED  STATE  variable  NORM 


3.1  Introduction 

As  mentioned  in  last  Chapter,  an  implicit  assumption  we  made 
was  that  all  directions  in  the  observation  spaces  are  equally  likely 
to  occur  with  unity  magnitude.  Typically,  however,  this  is  not  true 
in  practice  and  would  lead  to  undesirable  designs  in  some  situations. 
This  can  be  shown  by  a  simple  example. 

Consider  a  special  case  where  the  C^  are  invertable  matrices, 

e.g. 

=  I  I  -  i— 1,0,+1 

L1+0-2i  o.oi J 

2  2  • 

In  this  case  R(C.)=R  .  Because  0  (Z,  G)  represent  the  maximum 

i  max  i  * 

2 

projection  of  unit  vectors  in  Z^  on  G,  if  Z^  is  the  whole  space  R  , 

2 

o  (Z.’G)  will  always  be  one  no  matter  what  G  is  chosen.  Therefore 
max  i 

(2.19)  and  (2.22)  obviously  become  meaningless.  Formulation  (2,27) 
also  leads  to  a  poor  choice  as  we  will  see  later  on. 

The  reason  for  the  difficulty  with  (2. 19) and  (2,22)  is  that  in 
deriving  them  we  have  effectively  made  the  assumption  that  x  is 
equally  likely  to  lie  in  any  direction  and  take  on  any  magnitude. 
With  this  assumption  y  can  lie  in  any  direction  in  R(C^)  with  no 
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component  being  more  likely  to  have  a  large  value  than  any  other. 

In  this  Chapter  we  shall  develop  several  formulations  which 
are  parallel  to  Problem  (2.19) ,  (2.22)  and  (2.21)  but  which  take 
into  account  the  fact  that  we  may  have  some  information  about  the 
system  state  and  thus  about  likely  observation  values.  It  turns  out 
that  these  formulations  are  not  only  easier  to  solve  in  terms  of  the 
original  system  parameters,  but  they  also  give  useful  answers  for 
cases  such  as  when  C.  have  full  rank  as  shown  above. 

i 


3.2  Unknown  But  Bounded  State  Constraints 

As  mentioned  in  Section  3.1,  in  practice  we  always  have  some 
information  about  the  state  variable  x.  Typically,  we  at  least 
know  that  the  norm  of  x  cannot  be  arbitrarily  large.  Thus  we  may 
suppose  that 


(3.1) 


*11  J  *  N 


where  |]  xjj  =  x'Qx,  and  Q=Q'  is  a  specified  positive  definite  matrix. 
What  (3.1)  says  is  that  x  belong  to  an  ellipsoid  centered  at  the 
origin.  This  idea  is  identical  to  that  of  unknown-but -bounded  variables 
in  [6]  where  more  detailed  discussions  are  given  in  the  context  of 
problems  other  than  the  one  considered  here.  The  assumption!!  xjj 
effectively  allows  us  to  include  realistic  statements  concerning 
the  relative  magnitude  of  components  of  x  (and  thus  of  y  as  well) . 


; 

"i 


A 
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Note  that  one  logical  choice  for  x  is  its  covariance.  For  example, 
suppose  x  actually  satisfies  a  linear  equations 


x  =  A.x.  +  w. 
k+1  1  k  k 


where  w^  is  a  white  noise  process  with  covariance  E.  In  this  case 
could  be  thought  as  the  solution  of  the  Lyopunov  equation 


Qi  -  Wi +  E 


Now,  let  us  consider  again  the  problem  of  the  previous  section. 
__  2 

Because  y=C^x  and||  x  |j^M,  it  is  natural  to  modify  (2.17)  as 


(3.2) 

Let 

(3.3) 


min  max  max  ||P  C.x| 

G  1  IWg*M 


Q  =  P’P 


,  i=l ,2, . . . .  ,t 


Namely  P  is  any  square-root  of  Q.  Let  G  be  matrix  whose  columns 
form  an  orthonormal  basis  for  G.  Considering  (2.17)  in  this  section, 
we  have 


max  ||p  C.xi|2  =  max  |jGG'C,x||2 
x'Qx^M  1  x’Qx^M  1 


=  max  x'p' (p,)‘1c:gg,c.p“1px 
x’P’Px^M  1  1 
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=  max 
z  *  z^M 


z' (p*)  1c:gg,c.p“1z 
1  1 


=  max 
z '  z£l 


Mz'  (p'1)  •c:gg*c.p'1z 
1  1 


=  M  a  2((C.P~1) 'G 1 
max  l 


Thus  (3.2)  becomes 

(3.4)  min  max  0  2[  (C.p-1) *G] 

G ' G=I  i  maX  1 

2  -  -l 

which  is  the  counterpart  of  problem  (2.19).  Because  0  [(C.P  )  *G] 

max  l 

can  be  interpreted  as  the  maximum  projection  of  vectors  in  whose 

corresponding  state  variable  x  belong  to  an  ellipsoid,  (3.4)  can  be 

thought  of  as  minimizing  the  maximum  projections  of  all  such  vectors 

in  Z  . . ,2  .  Using  similar  arguments  we  can  also  obtain  the 

"*■  P 

counterpart  of  Problem  (2,22) 

2  -1 

(3  5)  min  £p.  cr  [(C.P  ) 'Gj 

G  G=I  1=1 

which  has  the  same  interpretation  as  that  of  (2.22).  Specifically, 
if  Q=I,  i.e,  the  restriction  on  x  is  ||  x||2  £M,  we  have 


(3.6) 


min  max 
G ' G=I  i 


2 

o 

max 


(C!G) 

1 


"X~" 
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and 

(3.7) 


t 


min  E  p . 
G'G=I  i=l  1 


2 

o 

max 


(C!G) 


Comparing  (3 . 4)  —  (3 . 7)  and  (2,19) ,  (2.22) ,  we  see  that  the  only 

difference  between  the  two  group  formulations  is  that  instead  of  Z^, 

C.P  1  is  used  in  (3,4)and  (3.5)and  C.  is  used  in  (3.6)  and  (3.7). 
l  l 

As  we  shall  see  in  the  example  at  the  end  of  this  Chapter,  if  C^P  1 
is  well-conditioned  for  all  i,  the  formulations  in  this  and  in  the 
previous  chapter  yield  similar  results.  If  this  is  not  true,  there 
may  be  a  significant  difference  between  the  results.  Note  also  that 
the  formulations  in  this  Chapter  require  only  the  determination 
of  c  ,  which  is  directly  related  to  the  A^  and  c^  and  thus  is  far 
simpler  to  calculate  than  . 

Continuing  our  development,  we  can  also  parallel  the  formulation 
given  in  eq.(2.27),  which  has  the  same  advantages  in  terms  of  compu¬ 
tational  complexity  —  i.e.  its  solution  involves  a  singular  value 
decomposition  rather  than  a  nonlinear  optimization  problem.  Speci¬ 
fically,  consider  the  problem 

t  , 

(3-8)  min  E  p.  }] (C  P  )  ' G {L. 

G'G=I  i=l  1  1 

where  the  p^  are  weights  which  can  be  interpreted  as  probabilities. 


Its  solution  is  (see  Eq.  (2.31)  and  (2.32)  in  Section  3.2) 
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(3,9) 


min  l  H (CP-1) 'g|£  =  Z  y2 
G ' G=I  i=l  i=l 


where  the  can  be  thought  of  as  the  normalized  matrices  including 

the  weights  p.  and  Y, ,  Y  are  the  smallest  s-r  singular 
1  l  s-r 

values  of  the  matrix  C  , 

d 

(3.10)  C  '  = 

d 


The  optimal  solution  for  G  is 
(3-11)  G*  =  I9l . gs_rl 

where  '<3s_r  are  the  left  singular  vectors  of  Cg  corresponding 

to  Y1 . Vr* 

The  interpretation  of  (3.8)  is  similar  to  that  of  (2,27).  That 
is,  as  shown  in  the  appendix  to  Chapter  2,  the  G  which  minimizes 

i 1  -  —  2 

I  G'C.P  subject  to  G'G=I  is  the  same  G  which  minimizes 

'  l  F  s-r 

-2- 

det(G'C^p  C^'G)  which  has  the  interpretation  as  minimizing  the 
volume  of  the  projection  of  possible  extended  observation  vectors 
on  G. 

As  a  final  comment,  note  that  and  can  be  related  through 

C  .  =  7,  T . 
l  l  1 


where  T.  is  a  full  rank  matrix.  Therefore  (3.8)  can  be  rewritten  as 


50 


(Suppose  P=I) 

t 

(3.12).  min  l  p.  ||(Z,  T . )  *G [ j  _ 

_  .  ,  l  ll  "  F 

G  1=1 

If  T  =1  then  (3.12)  becomes  the  same  as  (2.27).  This  means  that  (2,27) 
i 

is  a  special  case  of  (3.8).  In  the  same  way  we  see  that  (3.4)  and  (3.5) 

are  also  the  special  cases  of  (2.19)  and  (2,22)  respectively. 

In  this  section  we  have  derived  three  basic  formulations  under 

the  assumption  of  constrained  state  variable  norm.  This  approach 

allows  us  to  take  into  account  a  priori  information  about  the  relative 

sizes  of  the  state  variables.  Also,  the  dimension  of  the  resulting 

parity  check  (i.e,  of  the  space  G )  becomes  a  design  parameter  rather 

than  being  determined  by  the  dimensions  of  the  Z^, 

Note  that  from  the  derivation  of  the  formulations  in  this  Chapter 

we  see  that  the  results  of  the  noise  version  such  as  (2,50),  (2.52) 

and  (2.54)  can  easily  be  extended  to  the  results  in  this  Chapter  if 

we  use  C.  instead  of  Z.  in  (2.50)  and  (2.52)  and  C  instead  of  Z  in 
ii  a 

(2.54)  . 

In  next  section  we  will  give  a  further,  more  detailed  comparison 
between  the  formulations  of  this  Chapter  and  those  in  Chapter  2.  We 
will  focus  this  discussion  completely  on  the  noise-free  case  in  order 
to  make  our  points  more  clearly. 
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3,3  Comparison  Between  the  Two  Groups  of  Formulations 

So  far  we  have  obtained  two  groups  of  formulations  aimed  at 
solving  problem  (1.2).  One  set,  which  consists  of  (2,19) , (2.22)  and 
(2.27)  is  based  on  the  assumption  of  unconstrained  state  variable 
norm,  while  the  other  which  consists  of  (3,4),  (3.5)  and  (3.8)  is  based 
on  the  assumption  of  constrained  state  variable  norm.  In  order  to 
facilitate  further  discussion,  we  refer  to  these  as  groups  A  and  B 
and  list  them  as  follows. 


(A^) 


(A2 ) 


(A3) 


(Bl) 


(B2) 


(B3) 


min  max  0  (Z[G) 

,  max  i 

G  G=I  x 


min  I  0  (Z|G) 

.  max  i 
G  G=I  1=1 


min  I  IIz’gIL 
G ' G=I  i=l 


2  -  -1 

min  max  0  {(C.P  ) ' G ] 

,  ,  max  l 

G  G=I  i 


2  -  -1 

min  E  0  [(C.P  ) 'G] 

.  .  max  l 
G  G=I  i=l 


min  Y.  )|(C  P  L)  '41" 
G'G=I  i=l 
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LL 


The  main  appealing  feature  of  group  A  is  its  geometric  interpret 
tation  as  discussed  in  Chapter  2.  The  basic  concept  behind  these 
formulations  is  the  angle  between  two  subspaces,  a  quantity  which 


measures  the  closness  between  two  spaces.  Group  B  is  based  on  more 
realistic  assumptions,  i.e.  the  norm  of  the  state  variable  cannot 
be  arbitrarily  large,  which  makes  it  useful  in  some  situations  where 


group  A  becomes  useless  as  we  shall  see  in  next  example. 

Let  us  consider  again  the  example  mentioned  in  Section  3,1. 


In  this  simple  example 


1.0  0 


0.8  0.01 


and  by  Grand-Schmitt  orthogonization 


1.0  0 


1.2  0.01 


-0.62 

0.78 


Fig.  3.1 


0.64  -0,77 


0.77  0.64 
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As  we  point  out  in  3,1,  (Al).  and  (A2)  axe  meaningless  for  this  example. 
To  see  that  (A3)  also  provides  a  useless  answer  in  this  case  but  that 
the  Group  B  methods  yield  meaningful  results,  consider  methods  (A3) 
and  (B3 )  when  we  choose  G  to  be  a  one  dimensional  subspace. 


9 


Then  using  formulation  (A3)  and  (B3)  we  have  Table  3.1 


* 

f 

9 

/.  2 
1-g 

A3 

1.99 

0.993 

0.121 

B3 

0.04 

-0.714 

0.699 

Table 

3.1 

* 

where  f  denotes  the  minimum  value  of  the  criterian  being  minimized 
(in  (A3)  and  (B3)  respectively) .  The  results  are  quite  different  and  the 
one  from  B3  is  the  reasonable  one  the  reason  for  which  can  be  explained 
as  follows. 

In  this  example,  if  we  write 


x(k) 


x1  (k) 
x2  (k) 


Then 


\ 


y(k) 


=  C.x(k) 
1 
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x1  (k) 

(l+0,2i)x1(k)+0.01x2(k) 


If  the  norm  of  x(k)  is  constrained  to  be  less  than  M,  then  the  effect 
of  x^(k)  on  y(k)  is  far  more  important  than  that  of  x^fk).  Consequently, 

f1  1 

y(k)  is  more  likely  to  lie  along  I  and  ignoring  x  (k) 

l_l+0.2ij 

will  not  cause  significant  errors.  This  can  be  seen  from  Fig.  3.1 
where  the  observation  vector  y  is  likely  to  lie  in  the  shaded  area. 

It  is  obvious  the  G  obtained  in  B3  (see  Table  3.1)  is  the  reasonable 
one  while  the  G  resulted  from  A3  does  not  make  sense.  The  reason  for 
this  is  that  the  formulations  in  group  A  consider  all  directions  in 
to  be  equally  weighted  and  in  this  case  this  translates  into 
assuming  that  x2(k)  is  likely  to  have  a  magnitude  100  times  that  of 
x^ (k) . 

The  previous  example  illustrates  the  fact  that  if  C^P  ^are  very 
close  to  singular,  group  B  will  give  a  reasonable  solution  while 
group  A  will  not.  In  the  next  example  we  will  show  that  if  are 
well-conditioned  then  the  results  of  group  A  and  group  B  are  quite 
close.  In  this  example  the  system  considered  is  scalar: 

x(k+l)  =  a(n)x(k) 


y(k)  =cx(k) 
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where  0^1,2, 3  and  a(l)~0.8,  a(2)=1.2  ,  c=l.  Suppose  p=2,  then 


”  1.0  " 

r  i.o  i 

1.0  ' 

-  0.8 

r 

S  = 

o 

H 

_ _ 1 

f 

53  = 

1.2  . 

Normalizing  C^,  we  have 


"  0.7808  ‘ 

'  22  = 

'  0.7071* 

Z3  = 

"0.6402* 

.  0.6247  _ 

_  0.7071. 

0.7682  . 

Because  in  this  example  s=2,  r=l,  we  see  that  G  is  a  one  dimensional 
2 

subspace  in  R  .  Suppose  the  unit  vector  generating  G  is 


Then 


a  2  (z  !g)  =X  (g'z.zig) 

max  i  max  l  l 


=  Ung  + 


where  and  z^2  are  entries  of  .  Similarly 

o  2 (C !G)  =  (C..g  +  C.„ 
max  l  ll  l2 


A.  ^ 


-g  )' 


where  c.,  and  c . .  are  entries  of  C . . 

ll  i2  i 

2  -  2 

Because  of  the  simple  form  of  0  (C. 'G)  and  0  <Z.G)  for  this 

max  i  max  i 

low-order  example,  the  various  problems  can  be  solved  without  non¬ 


linear  programming. 
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To  solye  Q\l)  simply  observe  that  only  Z^.Z^  should  be  taken 
into  account.  The  minimum  is  obtained  at 


O 

max 


2 


(Z.  *G)  =  0 
1  max 


<Z2«G) 


Formulation  (A2)  is  simply  an  algebric  equation  for  g.  So  that  it  can 
be  solved  by  elementary  calculus.  To  solve  (A3)  we  observe  that  the 
matrix  Z  is 


— 

Z11  Z21  Z31 

Z12  Z22  Z32 


Using  a  singular  value  decomposition  subroutine  we  can  find  the  minimum 
singular  value  and  corresponding  singular  vector  which  is  the  desired 
G  in  our  simple  case.  Note  that  (Bl),  (B2)  and  (B3)  are  solved  in 
analogous  fashions. 

* 

The  results  are  shown  in  table  3.2  and  3.3  where  f  is  the  minimum 
value  given  by  each  formulation 


* 

/,  2 

f 

g 

1-9 

A1 

0.01112 

-0.7035 

0.7106 

A2 

0.02026 

-0.7024 

0.7118 

A3 

0.02026 

-0.7024 

0.7118 

Table  3,2 


B1 


0.02 


-0,7071 


0,7071 


B2 

0.0397 

-0.7118 

0.7024 

B3 

0.0397 

-0.7118 

0.7024 

Table  3.3 


From  Table  3.2  and  3.3  it  can  be  observed  that  the  resulting  detection 
spaces  G  are  almost  the  same.  This  is  due  to  the  fact  that  the  C^p  * 
matrices  are  well-conditioned. 
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CHAPTER  4 


EXAMPLE  OP  FORMULATION  (B3) 


4 . 1  Introduction 

In  this  Chapteran  example  of  a  three  machine  power  system  [11] 

with  realistic  data  is  analyzed  in  order  to  show  how  the  formulation 

(B3)  can  be  used  to  handle  the  problems  in  practice, 
tfl 

This  5  order  example  shows  how  a  simple  singular  value  decom¬ 
position  subroutine  is  used  to  solve  foumulation  (B3) .  The  norm  of 
the  maximum  projection  of  the  observation  vector  on  G  monotonely 
increases  with  increase  of  dimension  of  G  in  order  to  keep  the  maximum 
projection  of  the  normal  system  less  than  a  prescribed  value. 

Also  this  example  illustrates  how  different  values  of  p  i.e. 
different  numbers  of  lagged  outputs  in  extended  observation  vector, 
and  different  C  matrices  influence  the  results. 

In  Section  4,2  a  brief  discussion  of  the  three  machine  system 
is  given  in  order  to  explain  the  physical  meaning  of  the  system 
parameters  and  the  uncertainties.  Section  4.3  shows  the  results  for 
different  C  matrices.  Finally  in  Section  4.4  a  discussion  is  given 
to  suggest  how  to  choose  the  observation  space  in  practice. 
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4,2  Three  Machine  Power  System 

The  three  machines  in  this  power  system  are  coupled  to  each  other. 
The  whole  system  can  be  linearized  as  a  5**  order  continuous  time 
system  [11]  , 


x(t)  =  Fx(t) 
y(t)  =  Cx (t) 


where 

(4.2) 


x  ( t ) 


'Ao)r‘ 

A6C 

Aw 

L^d. 


with  Aw  ,Aoj  andAu.  being  the  relative  angular  velocities  of  the 
r  c  d 

generator  shafts  with  respect  to  a  reference  and  A6c  and  A6^  the  relative 
angles.  The  F  matrix  in  (4.1)  is 


11 

.00756 

.00486 

.00733 

-.00181 

0 

377 

0 

0 

0122 

f32 

1 33 

.0304 

-.00454 

i 

0 

0 

0 

377 

.292 

.163 

-.0292 

f54 

f  55 

where  f  ,£33  an<3  f,.,.  are  the  damping  factors  whose  values  are  in  the 
range  from  -.15  to  -.2,  and  f  and  f  are  spring  coefficients 
whose  values  are  not  known  precisely  and  can  change  from  -.1  to  -.4. 
The  constant  value  377  in  F  is  the  angular  frequency  of  the  60  Hz 


"V 


current  and  is  obviously  known  perfectly. 

In  this  example  we  consider  two  C  matrices. 


1  0  0  0  0 


0  0  10  0 


0  0  0  0  1 


1  0  0  0  0 


0  0  10  0 


0  1  0  0  c 


1  2 
Matrix  C  incorporates  the  observation  of  Aw^Ao^  and  Aoj^  and  C 

includes  the  observation  of  A^  ,Aw  and  A<$  . 

r  c  c 

In  order  to  apply  formulation  (B3)  to  this  example,  Eq.  (4.1) 
must  be  discretized  as  a  discrete  time  system 


x  (k+1)  =  Ax (k) 


y(k)  =  Cx(k) 


where 


A  =  exp(FA) 


and  A  is  the  sampling  interval. 

Because  the  fastest  angular  frequency  in  any  mode  of  this  system 
is  approximately  6.09  [11],  we  choose  A=0.25s  which  is  roughly  1/4 
the  period  of  the  fast  mode. 

In  next  Section  we  apply  formulation  (B3)  to  this  example  and  use 
singular  value  decomposition  to  obtain  the  optimum  detection  space 
corresponding  to  different  value  of  p  and  the  two  C  matrices. 
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4,3  Solution  of  Formulation  (B3) 

As  discussed  in  Section  4.2,  we  assume  the  uncertainties  only 

appear  at  eitries  f , , , f , f , f and  fr.  of  F  and  we  know  the  ranqes 
11  JJ  bb 

of  values  in  which  these  entries  may  lie.  As  mentioned  in  Chapter  2, 
Section  2.3,  in  order  to  apply  (B3)  to  this  example  we  must  first 
discretize  the  uncertainties  by  assuming  several  "extreme  points" 
of  matrix  F.  Here  we  assume  U--  (l,2,3)  (see  (2,16),  Section  2.3) 
and  the  corresponding  values  an<^  ^54  ta^e  on 

are  shown  in  Table  4.1. 


fll 

f  32 

f33 

Hi 

Ln 

f55 

-.2 

-.1 

-.2 

-,1 

-.2 

-.15 

-.4 

-.15 

-.4 

-.15 

-.15 

-.2 

-.15 

-.2 

-.15 

Table  4.1 

The  next  step  of  our  procedure  is  to  compute  F^,  A^=exp(F^A) 
and  the  corresponding  C^,  i=l,2,3. 


1  2 

where  C  can  be  equal  to  either  C  or  C  . 


In  order  to  investigate  the  influence  of  p  and  C  we  consider 
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(4.11) 


S(k> 


A 
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where  r_,  if=l . . s  are  the  singular  values  of  C  ordered  from 

1  3  3 

smallest  to  largest.  Eq.(4.11)  gives  the  minimum  value  of  T.  I1  C.*c" 

i=l  1 

when  the  dimension  of  G  equals  k.  Using  a  singular  value  decomposition 
subroutine  we  can  compute  r  .. i=l f s ,  the  corresponding  singular 
vectors  and  also  S(k),  k=l,..,..fs  for  the  four  different  cases.  The 
results  are  shown  in  Fig. 4.1  where  we  have  used  a  logarithmic  scale 
for  s (k) . 


4.4  Discussion 

3  _  2 

As  we  expected,  S(k)=  min  Z  j|  C . * G 1 )  is  a  monotone  function 

G‘G=I  i=l  1  F 

with  respect  to  k.  This  can  be  seen  from  Fig.  4.1,  although  it  is  more 
easily  seen  if  we  use  a  linear  scale  for  S(k).  This  is  done  in  Fig. 4. 2. 
From  Fig.  4.1  and  4.2  it  can  be  observed  that  in  order  to  keep  the 
projection  of  the  normal  system  sufficiently  small  the  dimension  of 
the  detection  space  should  be  smaller  than  7  or  8  in  case  2,3  and  4 
and  smaller  than  10  in  case  1,  because  S(k)  will  increase  very  fast 
if  k  exceed  these  numbers. 

This  example  illustrates  how  our  method  can  be  used  to  compare 
different  sensor  conf igurations  (C)  and  different  length  lags  (p) 
in  the  residual  generation  process.  Also,  for  a  given  C  and  p,  a 
curve  as  in  Fig.  5.2  provides  a  useful  visualization  of  the  effective 
redundancy  in  the  system.  For  example,  if  the  curve  has  a  dramatic 
"knee”,  i.e.  a  point  at  which  the  slope  of  the  curve  increases 
significantly  (as  it  does  for  all  four  of  our  cases),  one  has  a  clear 
indication  of  how  many  independent  parity  checks  can  be  made  reliably. 


case  2 


case  3  84 


case  1 


Fig.  4.2 
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CHAPTER  5 


FREQUENCY  DOMAIN  DESCRIPTION  OF 
THE  SPACE  OF  ALL  PARITY  CHECKS 


5.1  Introduction 

So  far  we  have  considered  the  failure  projection  method  using 

a  geometrical  approach.  The  principal  concepts  in  this  approach  are 

those  of  parity  checks  and  the  detection  space.  As  defined  in  section 

2.1,  the  observation  space  Z  ^  is  a  subspace  spanned  by  the  extended 

observation  vector  y  (k)  where 

P 

(5.1)  y  ’ <k)=[y’ (k)  y* (k+1)  . y'(k+p-l)]' 


Assuming  there  are  no  uncertainties,  the  detection  space  G^  is  the 

orthogonal  complement  of  Z  .We  also  defined  a  detection  vector  a 

P  P 

to  be 

(5.2) 


a  '  =  [a  '  a  ' 
P  PO  pi 


a  ’  ] 
PP 


which  satisfies 


(5.3)  a  'y  (k)  =  0 

P  P 

for  some  p=0,l, .  and  k=0,l, .  (  here  we  are  using  a  different 


symbol  for  detection  vectors  than  that  used  in  Chapter  2) .  Recall  that 

relationship  (5.3)  is  called  a  parity  check.  It  is  obvious  that  a  £  G 

P  P 
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and  finding  all  possible  parity  checks  of  length  p  is  equivalent  to 
finding  a  basis  for  detection  space  G  . 

In  this  Chapter  we  shall  develop  a  frequency  domain  approach 

(  in  the  case  of  no  system  uncertainty)  which  will  answer  the  question 

of  parity  checks  with  shortest  lengths  from  which  all  possible  parity 

checks  of  all  possible  length  can  be  generated.  In  Section  5.2  we 

shall  introduce  a  frequency  domain  description  for  parity  checks  by 

associating  a  polynomial  row  vector  with  each  parity  check.  These 
polynomial  detection  vectors  span  the  left  null  space  of  the 

polynomial  matrix  [C'(ZI-A)]'.  Thus,  finding  a  polynomial  basis  for 

this  left  null  space  is  equivalent  to  generating  all  possible  parity 

checks.  In  Section  4.3  we  further  show  that  among  all  polynomial  bases 

for  this  null  space  there  is  a  minimal  basis  consisting  of  parity 

checks  of  minimum  order.  Also  a  method  called  searching  the  crate  by 

rows  [7] , [3]  is  utilised  to  generate  this  minimal  basis.  Thus  to 

generate  all  posssible  parity  checks  the  following  procedure  may  be 

used  : 

1.  Generate  the  minimal  basis  m, (z) . . ,m  (z)  by  searching 

1  m 

the  crate  by  rows . 

2.  Form  a  mxm  polynomial  matrix  M(z)  whose  rows  are  the  minimal 

basis  m,  (z)  . . ,m  (z)  . 

1  m 

3.  Any  detection  vector  p(z)  can  be  generated  by  multiplying 

M (z)  on  the  left  by  an  arbitrary  polynomial  row  vector 

p' (z)  =  r' (z)M(z) 
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5.2  Frequency  domain  description  of  all  possible  parity  checks 

In  this  section  we  establish  the  link  between  the  time  domain 
and  frequency  domain  descriptions  of  the  parity  checks.  Then  we  show 
that  the  frequency  domain  description  of  all  possible  parity  checks  is 
nothing  more  than  the  left  null  space  of  the  polynomial  matrix 
[C  *  (ZI-A)  '  ]  '  . 

Recall  that  the  z-transform  of  a  sequence  y(k) ,  i=0,lr...,  is 

00 

(5.4)  y(z)  =  I  y(i)z  * 

i=0 

Using  this  definition  we  can  prove  the  following  theorem: 

Theorem  5.1  The  following  statements  are  eouivalent 


(5.5) 


(5.6) 


where  q(z) 


(5.7) 


1.  a  'y  (k)  =  0 
P  P 


2.  p ' (z) C (zI-A)  1  =  q’(z) 


is  some  nxl  polynomial  vector 


k=  1, 

p=0 , 1 , 


Proof : 

(1-^2):  The  z-Transform  of  (5.5)  is 

p  .  t-1 

E  a  'z  (y (z) -  E  y(r)z  r) 
t=0  r=0 


any  xQC 


R 


n 
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p  t  P  t  t-1 

=  I  z  a  y(z)-  E  z  a  *  E  y(r)z 
t=0  pt  t=0  pt  r=0 


p  trl 

=p'  (z)  y  (z)  -  E  a  £  E  y (r)  z  r  =  0 
t=0  p  r=0 


But 

(5.8) 


y(z)  =  C(zl-A)  xQz 


(5.9) 


y  (r)  =  Cx  (r)  =  CA  x 


0 


Therefore  the  z-Transform  of  (5.5)  is 

p  t-1 


(5.10) 


p' (z)C(zI-A)  ^x  =  E  a  '  E  Arzt  r 

0  t_0  P1  r=0  0 


Because  (5.5)  is  valid  for  all  x0ERn>  we  have  that 

-1  P  t"1  r  t-r-1 

(5.11)  p 1  (z) C  (zI-A)  =  E  a  *C  E  A  z  =  s'(z) 

t=0  r=0 


where  s(z)  is  a  polynomial  nxl  vector. 


(2-+1)  :  Define 


(5.12) 


g(k)  =  a  'y  (k) 
P  P 


P 

E 

i=0 


a  !y(k+i) 
Pi 


From  the  preceding  derivation,  in  particular  (5.10),  we  know  that 
the  z-transform  of  g (k)  is 


g  (z) 


p'  (z) C (zl-A) 


-1 


x0-s'(z)x0 


4 

1 
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Considering  (5.6) 


g(z)  =  [q1 (z)  -s'(z)]  xQ  all  xQtRn 

Because  q(z)  and  s(z)  are  polynomial  vectors,  g(z)  is  a  scalar  poly¬ 
nomial  function.  However 


g (z)  =  £  g(k)z~k 

k=0 

Therefore 

g(k)  «  ap»yp(k)  =  0  ,  k  =  1, .  .  . . . .  □ 


Since  (5.5)  is  nothing  more  than  a  parity  check 
thought  of  as  its  frequency  domain  description, 
a  link  between  the  frequency  and  time  domains. 
Next  we  shall  show  that  p(z)  and  q(z)  span 
of  the  polynomial  matrix  [C ' (zI-A) ' ) ' .  Here  the 
some  pxm  rational  matrix  H(z)  is  defined  [7]  as 
by  pxl  rational  vectors  v(z)'s  which  satisfy 


and  (5.7)  can  be 
this  theorem  sets  up 

the  left  null  space 
left  null  space  of 
a  subspace  spanned 


(5.13) 


v'  (z)H(z)  =  0 


Note  that  (5.6)  can  be  written  as 


p' (z)C  =  q' (z) (zI-A) 


Or 

(5.14) 


(p’ (z)  -q'(z)] 


C 

zI-A 


0 


Define  [p ’ (z)  -q'(z)]'  as  the  polynomial  detection  vector  and  denote 

the  left  null  space  of  [C (zI-A) l] '  as  N  .  Then  from  (5.14)  we  see 

lc 

that  the  polynomial  parity  vector  [p1 (z)  -q*(z)]'  e  N,  . 

lc 

Based  on  theorem  5.1  we  can  also  show  the  reverse  that  any 

polynomial  vector  on  N  corresponds  to  a  parity  check  of  some  length 

p.  Therefore  it  is  enough  to  know  one  polynomial  basis  of  N  in  order 

to  generate  all  parity  checks  of  all  possible  finite  length. 

In  next  section  we  shall  discuss  how  to  choose  a  specific  basis 

consisting  of  elements  of  shortest  length.  We  will  call  this  a 

minimal  basis  for  N,  . 

- lc 

5.3  Minimal  basis  of  N, 

lc 

A  rational  vector  space  has  a  fine  structure  which  is  associated 

with  its  minimal  polynomial  basis  [7) .  The  minimal  basis  is  a  basis 

with  minimal  order.  Namely  for  the  same  rational  vector  space  it  is 

impossible  to  find  another  basis  whose  vectors  have  the  same  or  smaller 

degrees  than  corresponding  vectors  in  the  minimal  basis  but  with  at 

least  one  vector  whose  degree  is  less  than  the  corresponding  one  in 

the  minimal  basis.  From  an  engineering  point  of  view  the  minimal 

basis  is  of  significance  because  from  (5.7)  and  (5.5)  we  see  that  a 

polynomial  detection  vector  with  lower  order  means  fewer  delay 

elements  are  required.  Also,  intuitively,  such  minimal  parity  checks 

should  be  the  least  sensitive  to  parameter  uncertainties. 

To  see  how  to  generate  the  minimal  basis  of  N.  let  us  first 

lc 
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introduce  the  concept  of  searching  the  crate  by  rows  [7]  ,  [8] ,  The  crate 
is  a  table  with  m  columns,  representing  the  rows  of  the  C  matrix 


c  1 . . ,cm'  anc^  n  rows  corresponding  to  the  powers  I,A,....,An  ^ 

So  the  (ijj)1"^  cell  of  the  crate  represents  the  row  vector  \ 

The  searching  process  is  as  follows.  We  first  search  the  first 
row,  i.e.  search  c  '  though  c  If  c . '  is  linearly  independent  of 

-L  ITJ 


D 

•  \  th 


c^', . ,Cj'l  '  Put  an  X  '*'n  cell.  Otherwise  put  a  0  in 

this  cell.  Then  search  the  second  row,  i.e.  search  c, ‘A  though  c  'A. 

l  m 

If  c . ‘A  is  linearly  independent  of  c,  '  . . ,c  ’ ,c  ' A, . . . . ,c . ' , A, 

1  1  ml  j-l 

then  put  an  x  in  (l.j)*"*1  cell.  Otherwith  put  a  0  in  that  cell.  We 
repeat  this  procedure  with  the  third  row  and  continue  in  this  way  with 
successive  rows  until  n  linearly  independent  vectors  have  been  found. 


Fig.  5.1  Example  of  crate  diagram 

m>=4,  n=5 


Note,  that  when  a  0  is  found  then  all  vectors  below  it  in  the  same 


column  will  also  be  linearly  dependent  on  the  previous  ones,  ^or 
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StepJj 

Consider  the  in  zeros  in  the  crate  diagram  (e.g.  Fig.  5.1). 
Suppose  one  zero  appears  at  the  (l,j)  cell  of  the  first  row.  From 
(5.15)  we  have 

(5.17)  [a1 . a  -1  0 _ 0] 


j-1 

=[  I  a.c!  -  c! ]x(k)=0 
.  ,  11  3 


Or 

(5.18) 

a'y(k)  =0 

where 

a'  =  [a^  . . 

3 j”!  ^  °- 

...0] 

Obviously  (5.18)  is  a  parity  check. 

Generally,  a  zero  in  the  j-th  row  would  represent  a  parity  check 
of  the  form 


?i 


x(k) 


(5.19) 


tai' 


.a . '  ] 
3 


y  (k) 


y  (k+j) 


=  0 


Its  corresponding  polynomial  parity  check  would  be 


(5.20) 


p' (z)y(z)  =  q* (z) 
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where 

j  i-1 

(5.21)  p' (z)  =  I  a.  *z 

i=l  1 

In  fact,  any  blank  cell  in  the  crate  diagram  is  also  corresponding 
to  a  linearly  dependent  vector  and  thus  to  a  parity  check  as  well. 
However,  as  we  mentioned  before,  it  can  be  generated  from  the  m  minimal 
ones. 


Step  2 :  Searching  the  crate  by  rows  generates  a  minimal  basis  of 


N 


lc' 


Proof : 

1.  Because  searching  the  crate  by  rows  generates  m  parity  checks, 
from  Theorem  5.1  we  know  that  they  correspond  to  m  polynomial  parity 
vectors  lpi'(z)  -q/(z)]  ,  i=l,..,.,m  and 


(5.24)  [p^'Jz)  -q^’tz)]'  e  N  ,  i=l,2,...,,m 

and  [a^  .  corresPont^:'-n<3  parity  check,  which  satisfies 


la’  ....  a!  ] 
il  ip 


y  (k) 


y (k+p-1 ) 


=  0 


2.  From  the  crate  diagram  we  can  easily  see,  that  every  cell 
except  those  with  x  represents  a  parity  check  or  equivalently  a 
vector  in  the  orthogonal  complement  of  the  range  space  of 
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In  fact,  the  collection  of  parity  checks  corresponding  to  these  cells 


span  the  or  ogonal  complements  of  the  range  spaces  of  for  all 
p=l,2, .  In  other  words,  they  produce  all  possible  parity  checks. 


Therefore  their  corresponding  parity  vectors  must  span  N^c*  But  we 
have  already  seen  that  these  parity  vectors  can  be  generated  by  the 
m  basic  ones.  So  the  m  basic  polynomial  parity  vectors  constitute  the 


basis  of  N,  . 

lc 


3.  From  (5.6)  we  see  that  the  order  of  p(z)  is  always  greater  than 
or  equal  to  that  of  q(z).  From  the  procedure  of  searching  the  crate 
by  rows  we  also  see  that  the  m  basic  polynomial  parity  vectors  have 
the  minimal  order. 

Combine  2  and  3  we  come  to  the  conclusion  that  the  m  basic  poly¬ 
nomial  parity  vectors  are  the  basis  of  N^c  with  minimal  orders. 
Therefore  they  are  minimal  basis  of  O 


From  the  development  in  this  section  we  have  the  following: 

Let  m, (z) . . (z)  denote  the  minimal  basis  and  let  p(z) 

1  m 

denote  the  detection  vector  corresponding  to  any  parity  check.  Then 
there  is  a  polynomial  vector  r(z)  so  that 


(5,25) 


p'  (z)  =  r ' (z)M(z) 


The  first  parity  check  corresponding  to  the  first  zero  in  the  first 
row  is 


a1'C  =[20  -1]C  =  0 


with  y  (k) ,y2 (k) ,y3 (k)  denoting  the  components  of  the  output  of  this 
system,  this  parity  check  is  given  by 


2y1(k)  -  y3(k) 


The  corresponding  polynomial  vector  (z)  is 


"2' 

0 

■? 

= 

0 

z  = 

0 

rl_ 

_-l 

Similarly,  by  a  simple  calculation  it  can  be  seen  that  the  second 
parity  check  corresponding  to  the  zero  in  the  second  row  is 


This  corresponds  to  the  parity  check 


YjOO  -  y2(k+l)  =  0 


The  corresponding  polynomial  vector  m2(z)  is 


1 

0 

Z  + 

0 

1 

0 

-1 

Z  = 

-z 

0 

0 

0 

-  - 

_  _ 

..  - 

By  simple  linear  algebra  we  can  show  that  the  third  parity  check 
corresponding  to  the  third  zero  in  the  third  row  is 


a  '  =  [-1  1  0  -2  0  0  -1  0  0] 


This  corresponds  to  the  parity  check 


-Y1 (k)  +  y2(k)  -2yi(k+l)  -  yx (k+2) 


The  corresponding  polynomial  vector  is 


Then 


m  '  (z) 

2 

0 

-1  ' 

m  '  (z) 

— 

1 

-z 

0 

2 

' (z! 

-l-2z-z2 

1 

0 

- 

- 

_ 

and  all  other  parity  checks  can  be  generated  as  in  Eq.  (5.25), 
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Conclusion 

In  this  Chapter  a  link  between  the  time  domain  and  frequency 
domain  descriptions  of  parity  checks  has  been  established.  Based  on 
this  link,  it  has  been  proven,  that  generating  all  possible  parity 
checks  is  equivalent  to  generating  a  basis  for  the  left  null  space  of 
the  polynomial  matrix  [C 1 (zT-A) ' ] ' .  Finally,  a  method  called  searching 
the  crate  by  rows  has  been  showm  to  nenerate  the  minimal  basis  for 
the  left  null  space  of  the  polynomial  matrix  IC 1  (zI-A)  1  ]  1  ,  i.e.  N  . 


i 

i 

t 

j 

i 

I 

i 

i 

\ 

i 

i 
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CHAPTER  6 

CONCLUSION 

6.1  Main  Contributions  of  This  Thesis 

The  main  contributions  of  this  thesis  are  as  follows. 

1.  The  formulation  of  the  Failure  Projection  Method  which 
provides  a  geometric  picture  of  the  problem  of  failure  detection  in 
the  presence  of  model  uncertainties  and  noise. 

2.  The  thorough  development  of  the  FPM  concept.  In  particular, 
two  groups  of  formulations  have  been  developed.  One  gives  distinct 
geometrical  interpretation  while  the  other  is  based  on  assuming  that 
one  has  available  a  priori  information  on  the  system  state.  Within 
two  groups  three  formulations  which  are  based  on  slightly  different 
criteria  and  have  decreasing  complexity  of  calculation  are  developed. 
The  simplest  require  only  a  singular  value  decomposition.  Also  two 
numerical  examples  are  given  which  show  the  relationship  among  these 

formulations  and  thus  provide  a  deeper  understanding  of  their  nature. 

3.  An  algebric  approach  has  been  developed  for  the  generation 

of  a  complete  set  of  minimal  length  parity  checks. 

4.  Extension  of  FPM  to  include  measurement  and  process  noise. 
Again  a  formulation  is  developed  which  only  involves  a  singular 
value  decomposition. 

5.  An  illustration  of  FPM  to  indicate  how  it  can  be  used  as  a 


design  tool  in  assessing  system  redundancy  and  in  determining  parity 
checks . 

6,2  Further  work 

We  feel  that  the  following  represent  the  most  important  directions 
for  further  work. 

1.  As  mentioned  in  Chapter  1,  in  this  thesis  we  only  examined 

Problem  (1.2)  -  minimizing  the  maximum  residuals  of  the  normal 

system  under  model  uncertainties  and  noise.  In  the  next  section  we 
briefly  discuss  some  possible  solutions  for  problem  (1.3),  i.e. 
finding  a  failure  detector  which  yields  good  performance  when  there 
is  only  one  postulated  failure  mode,  using  the  machinary  developed 

in  this  thesis.  The  completion  of  this  step  and  the  final  solution  of 
(1,3)  and  our  ultimate  goal  Problem  (1.1)  are  the  important  subjects 
for  future  work. 

2.  In  section  2.3  we  made  the  assumption  that  the  set  of  uncer¬ 
tainties  0  is  a  finite  set 


U  =  (1,2, .... ,t) 

and  pointed  out  that  it  is  a  unproven  conjecture  that  any  set  U  can 
be  replaced  by  a  finite  set  whose  corresponding  observation  subspaces 
are  the  "extreme  points"  of  the  original  set  of  observation  spaces 
as  the  parameter  vector  ranges  over  U.  This  conjecture  should  be 


proven . 
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3.  The  algebraic  approach  developed  in  Chapter  4  provides  a 

useful  starting  point  for  further  investigating  the  algebraic  aspects 

of  failure  detection.  For  example,  in  Chapter  4  we  only  considered 

parity  checks  of  finite  length  p.  But  in  practice  parity  checks  with 

infinite  length  are  often  used.  A  simple  example  of  a  infinite  length 

parity  checks  might  be  as  follows. 

Consider  a  system 


Therefore 


x^ (k+1) 

c 

1 

T 

x]L  (k) 

x2 (k+1) 

0 

1 

-  X2(k)  . 

’  y1(k)‘ 

x^  (k) 

y2oo 

x2  (k) 

C 

'  1 

0 

H-  o 

_ 1 

= 

1 

T 

CA 

_  0 

1  J 

It  can  be  seen  that  a  parity  ch>ck  would  be 


I  [x^k)  _  x^k)  +  Tx2(k)]=  0 


which  is  of  infinite  length. 
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6.  3  PPM  with  a  priori  information  about  one  failure  mode 

In  the  preceding  chapters  we  focussed  attention  on  Problem  (1.2) 
(see  Chapter  1,  Section  1.3)  without  taking  any  specific  failure  mode 
into  account.  In  this  section  we  briefly  consider  the  Problem  (1.3), 
where  one  specific  failure  mode  is  given.  To  solve  Problem  (1.3) 
we  must  find  a  residual  that  achieves  an  acceptable  tradeoff  between 
detection  and  false  alarm  characteristic.  As  mentioned  in  Section  1.3 
one  possible  criterian  for  this  problem  is  to  constrain  the  norm  of 
the  failure  residuals  when  the  system  is  unfailed  while  maximizing 
the  failure  residuals  when  the  system  fails. 

As  we  did  in  Chapter  2  ,  Section  2.2,  we  may  consider  the 
worst-case  situation.  Namely  we  may  constrain  the  maximum  projection 
of  the  extended  observation  onto  the  detection  space  G  to  be  less 
than  a  prescribed  quantity  when  there  is  no  failure,  while  maximizing 
the  minimum  pro jection  of  the  observation  vector  when  there  is  a  failure. 
This  leads  to  the  following  optimization  problem. 

max  min  min  j|  P  y||  2 
G  i  yeZf. 

II  v  ll=i 


under  constraints 

(6.1) 


G'G=I 

s-r 


max 


max 

zeZ  . 
ui 

II  z\\-l 


M 


l 
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where  Z  i=l,....,p  are  the  observation  spaces  of  the  unfailed 
ui 

system  and  Z  are  the  observation  spaces  of  the  failed  system.  Using 
the  same  arguments  in  Section  2.3  in  deriving  Eq.(2.18),  we  can 
rewrite  (6.1)  as 

max  min  a  .  (Z  ' G) 

_  .  mm  fi 


(6.2) 


G’G  =  I 

s-r 


max  0  (Z  !G)  <  M 
max  ui 


where  a  .  (•)  is  the  minimum  singular  value  of  some  matrix.  Instead 

mm 

of  considering  the  worst  case  one  can  imagine  the  possibility  of 
using  the  weighted  summation  or  expectation  as  condidered  in  section 
2.3.  Then  we  have 


P 

max  I  0  .  <Z  ’G) 
.  ,  mm  fi 
i=l 


(C2 )  G’G  =  I 

s-r 


P 

i  o  (z  :g)  <  m 

.  ,  max  ui 
i=l 


where  Z„.  and  Z  should  be  considered  as  the  normalized  version  of 
fi  ui 

Z_.  and  Z  .  as  we  did  in  Section  2.3  (see  formulation  (2.22a)).  In 
f  1  ui 

principle  (Cl)  and  (C2)  can  be  solved  by  nonlinear  programming,  although 


this  is  not  an  easy  task. 


Using  criteria  other  than  those  developed  so  far,  we  can  obtain 
other  formulations.  For  example,  if  we  choose  G  to  minimize  the  diffe 
rence  between  the  projection  norms  of  the  failed  and  unfailed  systems 


we  will  have  a  set  of  formulations  which  have  the  same  general 
objective  as  those  described  previously  in  this  section. 

As  one  example,  a  formulation  similar  to  (C 2)  would  be 


(D2) 


max 

G'G=I 


P 
[  E 
i=l 


a  . 

min 


<Zf!G> 


P 

E  a 

.  ,  max 
i=l 


(2uiG)  1 


Also  using  the  Frobenius  norm,(D2)  can  be  rewritten  as 


(D3) 


min  (||  Z  'g||  I  -II  Z _ * G 11  2  ) 

G'G=I 


where 


Formulation  (D3)  can  also  be  written  as 


(D3a) 


min  tr(G'ZSZ'G) 
G'G=I 


where 
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It  is  easy  to  see  that  (D3a)  can  be  solved  using  the  same  derivation 
we  used  to  solve  (A3)  and  (B3).  The  result  is 


s-r 

min  tr(C'ZSZ'G)  =  £  X. 

G ' G=I  i=l  1 


G*  =  [g  . g  ] 

1  s-r 

where  X^  are  smallest  s-r  eigenvalues  of  ZSZ*  and  g^, . rg  are 

corresponding  eigenvectors. 

Problem  (D3a)  can  be  solved  by  "S-singular  value  decomposition". 
If  we  denote 


Xs*°l  '  K-i°l  ■ 


. X  =c2  ,  X  =-o2  ,  , 

s-p+1  p  s-p+1  p+1 


X  =-o\ 

1  p+q 


then  we  know  that 


ZSZ'  =  u 


=UlSZU' 


where 
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E  = 


,  U’U  =  I 


If  E  is  nonsingular  we  have 


(E~1U,Z)S(Z'U£’‘1)  =  s 


Define 

V  =  E-1U'Z 

we  have 


VSV'  =  S 


V  is  called  S -orthogonal .  Therefore 
(6.3)  Z  =  UEV 

Then  the  solution  G  of  rank  r  is  obtained  by  taking  last  r  columns 
of  U.  The  thorough  examination  of  the  problem  and  approach  described 
in  this  section  remains  as  a  useful  direction  for  future  research. 
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